DOCUMENT RESUME 



ED 447 164 



TM 032 006 



AUTHOR 

TITLE 

INSTITUTION 

PUB DATE 
NOTE 
PUB TYPE 
EDRS PRICE 
DESCRIPTORS 

IDENTIFIERS 



Buck, Connie 

Summary of Outcome Data for Youth Alcohol and Other Drug 
Abuse Programs . 

Wisconsin State Dept, of Health and Family Services, 
Madison. 

1999-06-00 

31p. 

Reports - Descriptive (141) 

MF01/PC02 Plus Postage . 

♦Drinking; *Drug Abuse ; ^Evaluation Methods ; Grants ; 

* Prevent ion ; * Program Evaluation ; Tobacco ; * Youth Programs 
Wisconsin 



ABSTRACT 



The purpose of this report is to summarize steps taken in 
evaluating outcomes for the prevention of youth alcohol and other drug abuse 
(AODA) for selected AODA prevention programs and projects as reported to 
Wisconsin’s Department of Health and Family Services by grantees. A study by 
Wisconsin's Legislative Audit Bureau evaluated 6 projects, which funded 
approximately 60 grantees. In measuring outcomes, the evaluators looked at 
quarterly progress reports and the six AODA goals outlined by Wisconsin's 
governor: (1) reduce the number of youth smoking cigarettes; (2) reduce the 

number of youth using chewing tobacco; (3) reduce the number of youth using 
alcohol; (4) divert youth released from detention from AODA/drug selling; (5) 
decrease AODA for youth in counties receiving family preservation and support 
funds; and (6) other prevention goals. A total of 49 grantee files were 
reviewed, of these, 26 were not required to submit quarterly reports. Of the 
23 other grantees studied, all but one sent in at least one quarterly report, 
but only 5 initially reported outcomes tied to reducing youth AODA. To obtain 
additional information, evaluators requested additional 1998 outcome data. Of 
the 41 grantees that responded, 28 reported outcome data. Data are grouped by 
the goals addressed. (SLD) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



VO 



Summary of Outcome Data for Youth 
Alcohol and Other Drug Abuse Programs 




PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 



cs/ri 



/ CENTER (ERIC) 

Iv This document has been reproduced as 
received from the person or organization 
originating it. 




□ Minor changes have been made to 
improve reproduction quality. 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



1 



V 



/ 



(O 

o 

o 

CM 

CO 

o 



Department of Health and Family Services 
Office of Strategic Finance 
Strategic Planning and Evaluation Section 



I 



O 



ERIC 



2 



BEST COPY AVAILABLE 



Summary of Outcome Data for Youth 
Alcohol and Other Drug Abuse Programs 



Evaluation Section 
Connie Buck, Evaluation Analyst 
June, 1999 



O 

ERIC 



3 



Introduction 



The purpose of this report is to summarize steps taken in evaluating outcomes for the prevention 
of youth alcohol and other drug abuse (AODA) for selected AODA prevention programs/projects 
as reported to the Department of Health and Family Services by grantees. 

This report responds to a September 1996 audit of prevention programs by the Legislative 
Audit Bureau (LAB). Specifically, the Department is evaluating outcomes tied to the Governor’s 
goals for decreasing problems related to youth AODA. The Governor’s goals are used for 
reporting because this is consistent with federal directions for reporting outcomes for the Safe 
and Drug Free Schools and Communities Act. As a result of these federal reporting 
requirements, grantee’s quarterly reporting forms were revised requiring them to file quarterly 
reports with the Department, showing whether they are meeting these goals. The Governor’s 
goals for AODA prevention include: 

1 . Reduce the number of youth smoking cigarettes in the past six months. 

2. Reduce the number of youth using chev^ng tobacco. 

3. Reduce the number of youth using alcohol in past six months. 

4. Divert youth released from detention from AODA/drug selling. 

5. Decrease AODA for youth in counties receiving family preservation/support funds. 

6. Other. 

The LAB audit included the following six projects, which have the common goal of reducing 
youth AODA: 

■ Neighborhood Drug Use and Violence Prevention Project 

■ City-Wide/ Athletic Drug Prevention 

■ Community AODA Prevention 

■ Milwaukee AODA Education 

■ Safe and Drug-Free Schools and Community Act 

■ Prevention and Treatment Block Grants 

These projects fund approximately 60 grantees. Eleven tribes are funded through the Safe 
and Drug-Free Schools and Community Act; however, the tribes were not included in this 
evaluation because their funds are pooled together and dollars spent on prevention are not 
separated from those spent on treatment. In addition to the sixty grantees referred to above, we 
found three additional grants that were not on the original list provided by the Division. They 
were: Division of Health - HIV/AIDS Prevention; Word of Hope Ministries, Inc. - Family 
Resource Technology Center; and UW Systems, UW Extension-Milwaukee Family Project - 
Professional Development. No quarterly reports were found for these three grantees. 

Initial research included reviewing 1998 quarterly reports and application summaries within 
each grantee’s file. In measuring outcomes, we looked at the six Governor’s goals listed above, 
as foimd in the quarterly progress reports. We also added a category for decreasing the number 
of youth using drugs. For all of these goals, we looked for information indicating a change in 
knowledge, attitudes and/or behavior. 
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A total of 49 grantee files were reviewed. Over half of these (26 in total) were not required 
to submit quarterly reports. Most of those not required to submit quarterly reports (20 of the 26) 
were DARE projects that were not required to submit quarterly reports to DHFS because they 
were being evaluated by the Department of Public Instruction, and DHFS did not want to 
duplicate efforts. The other 6 were not required to submit quarterly reports for a variety of 
reasons. These included: projects with one-time funding for training or TA services, projects for 
which program transitions were occurring (switch from one vendor to another), projects for 
which funding for a program was ending due to federal funding cuts, or projects with small 
funding where the energy of preparing quarterly program progress reports was felt to be an undue 
burden that would take time from providing services. 

All but one of the other 23 grantees that were required to report sent in at least one quarterly 
report, but only five initially reported outcomes tied to reducing youth AODA. Of these five, all 
but one grantee reported their outcome information in the application summaries rather than in 
their quarterly progress reports. 

Three grantees reported changes in attitudes and increased knowledge of problems associated 
with drug use. The other two grantees reported the following data from pre/post test scores: 

■ 11% decrease in smoking among participants 

■ 7% decrease in marijuana use among participants 

■ 7% reduction in the number of youth using alcohol or drugs. 

Of the 22 grantees that filed at least one quarterly report, several listed information in the 
section on outcomes. Unfortunately, this information was often process data rather than outcome 
data. In addition, some application summaries indicated that the grantee was conducting some 
type of pre/post test and therefore, outcome data may be available. 

In order to obtain additional information, we sent grantees a letter requesting additional 1998 
outcome data. The letter provided a brief description of the difference between outcome data and 
process data. Forty-one grantees responded by telephone and/or mail. Of the 41 grantees that 
responded 28 reported outcome data. The combined results of our initial review of grantee files 
as well as additional data received in response to the letter that was sent is summarized below. 
Data are presented at the project level, under the following categories: 

Goal 1 : Reduce the number of youth smoking cigarettes in the past six months. 

Goal 2: Reduce the number of youth who use chewing tobacco. 

Goal 3: Reduce the number of youth using alcohol in the past six months. 

Goal 4: Divert youth released from detention from AODA/drug selling. 

Goal 5: Decrease AODA for youth in counties receiving family preservation/support funds. 

Other - Drug Use 

Other - Alcohol and/or Drug Use 

Other - Development of Positive Youth Behavior 

Other - Increased Knowledge 

Other - Miscellaneous 



Project Outcomes 



Neighborhood Drug Use and Violence Prevention Project 

Currently fourteen grantees receive funding under this project. Twelve of the fourteen grantees 
responded and eleven of twelve reported outcome data. Due to the nature of the project, some 
grantees reported outcomes that address violence prevention rather than youth AODA. 

Goal 1 : One grantee reported a 4% reduction in smoking in all grades except the 12*'’ grade 
where there was a 2% increase for females; Another grantee reported that 60% of participants 
demonstrated the ability to say no to a cigarette and 95% identified alternatives to smoking. 

An intermediate outcome related to this goal is increasing community awareness about 
selling cigarettes to underage people. One grantee checked all businesses for compliance of not 
selling cigarettes to underage people. In 1998, eight out of 37 businesses, or 22%, were found to 
be in violation. In 1 997, 33% of the businesses were found to be in violation. 

Goal 2 : One grantee reported both increases and decreases in the number of youth using 
chewing tobacco, varying by age group. Specifically, chewing tobacco increased by 2% for 12*'’ 
graders and decreased by 1% for 10*'’ graders. 

Goal 3 : One grantee reported a reduction in the number of youth using alcohol in the past six 
months. Another reported a 5% decline in alcohol use by 6"’ & 8"’ graders, a 2% increase for 1 0"’ 
graders and no change for 1 2"’ graders. 

An intermediate outcome related to decreasing the number of youth who use alcohol is 
increasing their knowledge of the effects of alcohol. Four grantees reported intermediate 
outcomes. One reported that 600 graduating seniors were made aware of the dangers of drinking 
and driving. The second reported that 95% of block parties and other neighborhood events are 
now alcohol free. The third reported that 60% of participants demonstrated the ability to say no 
to alcohol, 90% changed their attitude about the effects of drinking, and 95% identified 
alternatives to drinking. 

The fourth grantee reporting intermediate outcomes distributed 30,000 grocery bags with a 
message promoting no use of alcohol when pregnant. Understanding of healthy choices 
regarding no use increased by 50% for those who received the grocery bags with the prevention 
message. This grantee also provided Fetal Alcohol Syndrome (FAS)/Fetal Alcohol Effects 
(FAE) Inservice for a county agency and students at Fox Valley Tech College. Ninety-five 
percent of participants reported a better understanding of FAS/FAE and felt they were better 
equipped to serve children diagnosed with FAS/FAE as a result of this program. The grantee 
also held Post Graduation Alcohol Free Parties and estimated that 90% of participants did not use 
alcohol following graduation and that 80% did not drive under the influence. 

Goal 4: No outcomes reported, 



Goal 5 : No outcomes reported. 



Other - Drug Use : One grantee, whose focus is crime and violence prevention, reported that 900 
middle school students were able to identify ways to resist the temptation of drug use. Another 
grantee reported that 92% of youth reported no drug use. This information provides the grantee a 
baseline for measuring changes in drug use in their community. Another grantee reported that 
inhalant use has increased by 7% for 6"’ grade and decreased by 5% for 8"’ grade. In addition, 
LSD use has decreased by 3% for 8"’ and lO* grades. Finally, another grantee reported that 90% 
of youth were able to identify examples of peer pressure and verbalize “no” to becoming 
involved in drug activity. 

Other - Alcohol and/or Drug Use : One grantee reported a 7% decrease in the number of youth 
using alcohol or drugs. Another grantee reported that 2,400 elementary students understood 
AODA issues. A third grantee used funds to host an Alcohol/Drug Free Post Prom Event where 
85% of those attending reported they did not use alcohol and drugs and 45% reported they did 
not drive under the influence of alcohol or drugs. A fourth grantee reported a 5% decrease in use 
for 6"’ and 8'“’ graders. 

Other - Development of Positive Youth Behavior : One grantee reported a 30% reduction in the 
level of anger of participants. Another grantee reports that their Alternative to Gangs project 
(after-school activities) has realized and maintained a 95% success rate in diverting youth into 
positive activities. 

Yet another grantee reported that after hearing a guest speaker, 80% of students had a greater 
understanding of juvenile laws in relation to crime and consequences. They also reported that 
90% of participants in “Positive Choices,” a youth AODA holistic curriculum, were made aware 
of their self-esteem and ways to enhance it as well as setting goals. A fourth grantee reported 
that due to increased community awareness and involvement, Waukesha has seen an 8% 
decrease in reported index crimes during the first six months of 1998 and a 17% decrease when 
comparing 1997 to 1996. Anther grantee reported that after attending a Two-day Resiliency 
Training program, 75% of attendees (n=25 educators) reported a different attitude and used the 
research based approach in the classroom. 

Other - Increased knowledge : No outcomes reported. 

Other -Misc .: One grantee reported a 50% increase to the “pro-social” question, “I believe I 
need to make things right with others.” Another grantee reported 1,161 parents increased their 
awareness and communication skills for discussing drugs, alcohol, and violence with their 
children. 
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City -Wide/Athletic Drug Prevention 



The only grantee under this project, Career Youth Development, did not report any outcome 
data. 

Community AODA Prevention 

All six grantees responded to our inquiry, and five of them reported outcome data. One of the 
grantees reported that they use the Search Institute Developmental Assets Model for measuring 
the impacts of their programs. 

Goal 1 : One grantee reported that participant smoking decreased by 9% and participants who 
stated it was easier to say “no” increased by 1 8%. 

Goal 2 : No outcomes reported. 

Goal 3 : Two grantees reported outcomes for the third goal, reducing the number of youth using 
alcohol in the past six months. One grantee reported survey results that 85% of participants in 
the Juvenile Underage Drinking Offender (JUDO) Program intend to make lower-risk choices 
about alcohol in the future. The grantee noted that the JUDO Program has seen a large decrease 
in participation. Offenders are opting out of the'program and choosing to take suspensions 
instead of attending the JUDO Program. The other grantee reported a 19% decrease in alcohol 
consumption and those participants who stated it was easier to say “no” to alcohol increased by 
36%. 

Goal 4 : No outcomes reported. 

Goal 5 : No outcomes reported. 

Other - Drug Use : One grantee reported that marijuana use decreased by 10% and those who 
believe they risk harming themselves if they use marijuana increased by 12%. 

Other - Alcohol and/or Drug Use : One grantee reported that 100% of the 33 participants in the 
Milwaukee Youth Together Against Drugs program increased their knowledge base of Alcohol 
Tobacco and Other Drug Abuse (ATODA) and its harmful effects. Another grantee reported that 
73% of participants in various educational presentations (e.g. Red Ribbon Week, Drunk Driving 
Awareness) reported that they learned ways that their life could be negatively impacted upon if 
they drank alcohol or used drugs. 

One grantee reported the following responses: 

■ Percentage of violators whose understanding of drugs/alcohol; 
increased (75%) 
remained the same (15%) 
decreased (1.0%) 
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■ I did not leam anything new about alcohol, myself, or drugs in this class. 

0% strongly disagreed 
5% disagreed 

- 35% neither agree or disagree 
40% agree 

- 20% strongly agree 

■ I understand myself and my actions better because of this class. 

0% strongly disagreed 
25% disagreed 

30% neither agree or disagree 

- 40% agree 

5% strongly agree 

Other - Development of Positive Youth Behaviors : One grantee reported that between the 3'‘‘ 
and 4*'' quarter of 1998, there was a 61% increase in the number of youth who became involved 
in positive alternatives. Another grantee reported that 96% of participants (n=51) indicated they 
learned how to be a more positive team player and that their self-esteem increased after attending 
the “Youth Leadership/ATODA Training Conference.” Workshops included topics such as Teen 
Health on Teen Pregnancy, Self-esteem, Drinking, Driving and Consequences, and Stress 
Management. In addition, three youth attended the Summit of the State Prevention Conference 
and youth evaluations indicated that 100% of the youth gained information on healthy lifestyles. 

Another grantee reported that 90% of youth who were involved in planning the alcohol and 
drug free alternative activities held during 1998 reported that they felt they played an important 
role in implementing the activity. Fifty-two percent of the youth completing surveys at the 
alcohol and drug free alternative activities held in 1998 reported that these types of activities 
showed them that adults cared about how they use their time. Forty-two percent of the children 
participating in Germantown Kids Klub improved the degree in which they act on their thoughts 
and feelings without regard to the negative consequences. 

Other - Increased Knowledge : A grantee reported that 83% (n=l ,545) of parents and school 
personnel participating in “identification and referral training” successfully completed all 
components of the training and increased their knowledge base. In addition, 395 youth were 
recruited to become “peer ATODA prevention facilitators.” Sixty-five percent of these youth 
increased their knowledge base and facilitated peer rap sessions at various community-based 
organizations and events. 

Other - Miscellaneous : No outcomes reported. 
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Milwaukee AODA Education 



There is one grantee listed under this project, Social Development Commission, and they did not 
report outcomes in either their quarterly reports or in response to our inquiry. 



Safe and Drug^Free Schools and Communities Act 

This project funds a variety of grantees including: Wisconsin Regional Teen Institute (WRTI), 
Prevention Resource Center, Wisconsin Positive Youth Development, Milwaukee County-UW 
Extension-Family Support Project, Ethan Allen School, Tribal Youth and Drug Abuse Resistance 
Education (DARE). Three grantees, WRTI, Milwaukee County UW Extension-Family Support 
project and DARE, reported outcomes. 



Wisconsin Regional Teen Institute 

A report by Richard L. Schnell and Zoanne K. Schnell titled, “The National Prevention Study: 
Student Values, Behaviors and Knowledge. Research Monograph No. 97-201 Wisconsin 
Regional Teen Institutes Pretest-Posttest Change and Program Impact” provided outcome data 
for WRTI. The report shows the results of a pretest administered to 169 WRTI students in 1997. 
Sixty-three students completed a posttest assessment six months following participation in 
WRTTs program. Matched paired analysis of data was done for 47 students (75% of the posttest 
sample). This study reports on Sr. High school students. WRTI also sent a quarterly report in 
response to our inquiry that provided data for both Jr. and Sr. High school WRTI students. 
Following are their reported outcomes: 



Goal 1: Percentage of WRTI students who report smoking cigarettes. 



Jr High 


Sr High* 


Pretest 


Posttest 


Pretest 


Posttest 


5.3% 


8.0% 


14.9% 


19.6% 



Goal 2: Percentage of WRTI students who report chevwng tobacco. 



Jr High 


Sr High* 


Pretest 


Posttest 


Pretest 


Posttest 


0% 


2.7% 


8.5% 


6.5% 
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Goal 3: Percentage of WRTI students who report using alcohol. 



Jr High 


Sr High* 


Pretest 


Posttest 


Pretest 


Posttest 


12% 


17.6% 


46.8% 


39.1% 


Evidence based on pre-post comparison indicates no statistically significant changes in substance use. 



Other - Development of Positive Youth Behaviors : A pre-posttest comparison showed that Jr. 
and Sr. High WRTI students increased their participation in prevention activities and programs 
In addition, WRTI students were more likely to intervene in friends’, classmates’, and siblings’ 
drug abusive behavior. 



Milwaukee County UW Extension 

The second grantee to report outcome data under this project was the Milwaukee County UW 
Extension-Family Support project. Their outcome data included: 

Other - Development of Positive Youth Behaviors : A panel of teen mothers provided anecdotal 
information on how the support of their “home visitor” led them to set and work towards goals. 
Each teen mother had obtained childcare, returned to school and had plans for college. 

Other - Increased Knowledge : Ninety-five percent, or 320 “home vistors” who attended 
professional development and training rated the training as pertinent to their work with families. 
The training included a module on domestic violence and substance abuse. 



Drug Abuse Resistance Education 

The third grantee to report outcomes was the DARE program. DARE provides grants to 19 
sheriff and police departments who provide AODA education throughout Wisconsin school 
districts. In addition, funds for DARE officer training are allocated to the Department of Justice. 
Sixteen out of twenty DARE grantees responded to our inquiry and ten reported outcome data. 
Following is a summary of the DARE grantees’ responses. 

Goal 1 : One DARE grantee reported that in the past six months, the number of reported 
instances of youth smoking cigarettes decreased by 25%. During the first quarter, they only had 
one instance of cigarette use by students participating in DARE. Another grantee reported that 
0% of the 5“' graders smoked since the DARE program; however, they did not provide pretest 
data for comparison. Another reported that in the 1 997-98 school year, 9% of S'*" graders 
reported using cigarettes daily or within the last week. In the 1998-99 school year, that number 
was reduced to 6.16%. Another reported that at the completion of class, 60% of students 
indicated they did not believe they would be involved in smoking. 

Although one grantee was unable to report whether DARE had been effective in reducing the 
incidence of youth smoking, they reported baseline data that will be useful in future evaluations. 
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A reported intermediate outcome for Goal 1 is reducing the incidence of cigarette possession. 
One grantee reported that there were no arrests for minors possessing or attempting to buy 
cigarettes in the past six months; however, we do not know previous arrest rates for this violation 
and are unable to draw any conclusions about the effectiveness of the program. 

Two grantees combined Goals 1 and 2. One reported that yearly statistics indicated a 72% 
reduction in all tobacco citations among youth from 1997 to 1998. The other reported that 23% 
of the parents who responded to a survey agreed that “the DARE program increased youth 
awareness to the dangers of cigarettes and chewing tobacco.” A third grantee reported the 
number of citations issued for tobacco possession, but did not provide baseline data for 
comparison. 

Goal 2 : One grantee reported that in the past six months they did not have any reported instances 
of youth chewing tobacco. Another reported that 0% of 5"’ graders have chewed tobacco since 
DARE. Yet another reported that in the 1997-98 school year, 1% of 8"’ grade students reported 
using chewing tobacco. That number increased to 1.3% in the 1998-99 school year; however, 
both years were less than the 1996-97 school year. 

A reported intermediate outcome by one grantee for Goal 2 was that no arrests were made for 
possession of chewing tobacco during the past six months. 

Goal 3 : One grantee reported that “statistics show that the youth alcohol use is similar to 
previous data, however, other juveniles are informing the department about parties (we may be 
reaching more juveniles).” Another reported that 70% of DARE students could identify various 
health risks associated with consuming alcohol on a regular basis, a 57% increase since the 
DARE program began during the first week of January. 

Another grantee reported that 0% of 5"' graders used alcohol since the DARE program and 
that 100% of them realized that drinking alcohol every day will harm the body. 

One grantee reported that the total number of alcohol related crimes decreased, allowing for 
additional enforcement of youth alcohol related cases. Between 1997 and 1998, there was a 5,5% 
increase (from 65 to 101) in the number of citations issued to youth. Another grantee simply 
reported the number of citations issued for underage drinking, but no baseline data was available 
for comparison. 

Goal 4 : No outcomes reported. 

Goal 5 : No outcomes reported. 

Other - Drug Use : One grantee reported in their grant application that participants increased 
their knowledge about drugs and learned to say “no.” Another grantee reported the following 
responses from participants: 

■ 99% said that DARE helped them. 



■ 95% clearly understood the dangers of mind altering drugs, 

■ 98% felt they would remember the dangerous consequences of mind altering drugs, 

■ 94.5% are more likely to be involved in positive alternatives to drugs, 

■ 99% are glad they took a stand against drugs and violence. 

Another grantee reported that they have had no reported instances of youth in the DARE 
target area selling drugs in the past six months and they attribute this to the success of the DARE 
program. 

Another grantee reported that at the completion of class, 90% of the students indicated that at 
this point in their lives, they would say no to drugs. 

Other - Alcohol and/or Drug Use : One grantee reported no instances of AOD use by students 
participating in the DARE program during the first quarter. Another reported that district-wide 
in the 1997-98 school year, 49 referrals were made for AODA issues. In the 1998-99 school 
year, the number of referrals decreased to 34. Still another reported that 79% of the parents who 
responded to a survey agreed that the DARE program increased youth awareness of the dangers 
and consequences of alcohol and/or drug use. 

Other - Development of Positive Youth Behaviors : One grantee reported that 15% of students 
gave indictors that they had higher self-esteem after completing the course and 100% of the 
students had more contact with local law enforcement in a positive manner. Another reported 
that 93% of the parents who responded to the survey agreed that “overall, the DARE program 
was successful in educating students about substance abuse, peer pressure, gangs, and related 
topics.” 

Other - Increased Knowledge : One grantee reported data from recent surveys of 6"’ graders 
indicating that 97% of students could identify at least one negative consequence of using alcohol 
and drugs. 

Other - Miscellaneous : No outcomes reported. 



Prevention and Treatment Block Grants 

These funds are distributed throughout the previously listed grantees/projects, therefore, the 
results of the grants have been reported above. 
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